Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev/r18 training rehydration #39

Merged
merged 4 commits into from
Nov 1, 2024

Conversation

Purg
Copy link
Member

@Purg Purg commented Nov 1, 2024

What does this PR do?

Update with a few found fixes to get R18 training underway.

Purg added 4 commits November 1, 2024 10:55
* Fix pre-vectorization dataloader iteration. Found with some high
  values of dataset size, memory errors could occur. Using pinned memory
  in dataloader seems to fix this.
* Making use of the file-system sharing strategy for internal
  pre-vectorization, restoring current sharing strategy afterwards.
* Fix cache checksuming to include module and classname of vectorizer
  implementation being utilized.
* Moved internally used dataset for pre-vectorization out as a
  global-level class definition instead of being inside a closure.
* Updated "main" function a bit to be parameterized and spit out some
  useful stats for configuring a training run (number of windows,
  dimension of embedding vector).
@Purg Purg merged commit 4c25924 into PTG-Kitware:main Nov 1, 2024
0 of 11 checks passed
@Purg Purg deleted the dev/r18-training-rehydration branch November 1, 2024 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants